Introduction to Build Tools GN & Ninja

Overview

Generate Ninja (GN) is a new tool used by the chromium project to replace GYP. Since GN is written in C++, it is 20 times faster than GYP written in Python. The new DSL syntax of GN is also considered to be better written and maintained.

GN depends on Ninja. Ninja is a build system that pursues speed. Compared with other build systems, Ninja is characterized by being fast and concise, retaining only the fewest features to improve compilation speed.

Usage of GN

Add a .gn to the root directory of the source project, with the following content:

buildconfig = “//build/BUILDCONFIG.gn”

The directory where .gn is located will be identified by the GN tool as the source root of the project. The content of .gn is basically to use buildconfig to specify the location of the build config, where //build/BUILDCONFIG.gn is used to specify the path relative to the source root.

Step 1: Create /BUILDCONFIG.gn

According to the previous settings, you need to add another BUILDCONFIG.gn under build/ folder, the content is as follows:

1
2
set_default_toolchain("//build/toolchains:gcc")
cflags_cc = ["-std=c++11"]

The first line specifies the toolchain to be used. The parameter is a label. //build/toolchains:gcc refers to the gcc toolchain defined in build/toolchains/BUILD.gn. The second line is to set the command line parameters used when compiling C++.

Step 2: Create /build/toolchains/BUILD.gn

Because GN has no built-in toolchain rules, various tools in the toolchain such as cc, cxx, link, etc. must be specified by themselves, here is an example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
toolchain("gcc") {
  tool("cc") {
    depfile = "{{output}}.d"
    command = "gcc -MMD -MF $depfile {{defines}} {{include_dirs}} {{cflags}} {{cflags_c}} -c {{source}} -o {{output}}"
    depsformat = "gcc"
    description = "CC {{output}}"
    outputs = [
      "{{source_out_dir}}/{{target_output_name}}.{{source_name_part}}.o",
    ]
  }
  tool("cxx") {
    depfile = "{{output}}.d"
    command = "g++ -MMD -MF $depfile {{defines}} {{include_dirs}} {{cflags}} {{cflags_cc}} -c {{source}} -o {{output}}"
    depsformat = "gcc"
    description = "CXX {{output}}"
    outputs = [
      "{{source_out_dir}}/{{target_output_name}}.{{source_name_part}}.o",
    ]
  }
  tool("alink") {
    rspfile = "{{output}}.rsp"
    command = "rm -f {{output}} && ar rcs {{output}} @$rspfile"
    description = "AR {{target_output_name}}{{output_extension}}"
    rspfile_content = "{{inputs}}"
    outputs = [
      "{{target_out_dir}}/{{target_output_name}}{{output_extension}}",
    ]
    default_output_extension = ".a"
    output_prefix = "lib"
  }
  tool("solink") {
    soname = "{{target_output_name}}{{output_extension}}"
    # e.g.
    "libfoo.so".
    rspfile = soname + ".rsp"
    command = "g++ -shared {{ldflags}} -o $soname -Wl,-soname=$soname @$rspfile"
    rspfile_content = "-Wl,--whole-archive {{inputs}} {{solibs}} -Wl,--no-whole-archive {{libs}}"
    description = "SOLINK $soname"
    # Use this
    for {
      {
        output_extension
      }
    }
    expansions unless a target manually
    # overrides it(in which
        case {
          {
            output_extension
          }
        }
        will be what the target # specifies)
      .
    default_output_extension = ".so"
    outputs = [
      soname,
    ]
    link_output = soname
    depend_output = soname
    output_prefix = "lib"
  }
  tool("link") {
    outfile = "{{target_output_name}}{{output_extension}}"
    rspfile = "$outfile.rsp"
    command = "g++ {{ldflags}} -o $outfile -Wl,--start-group @$rspfile {{solibs}} -Wl,--end-group {{libs}}"
    description = "LINK $outfile"
    rspfile_content = "{{inputs}}"
    outputs = [
      outfile,
    ]
  }
  tool("stamp") {
    command = "touch {{output}}"
    description = "STAMP {{output}}"
  }
  tool("copy") {
    command = "cp -af {{source}} {{output}}"
    description = "COPY {{source}} {{output}}"
  }
}

Notice how the cflags_cc mentioned earlier is used by tool cxx.

1
2
3
4
5
tool("cxx") {
  ...
  command = "g++ -MMD -MF $depfile {{defines}} {{include_dirs}} {{cflags}} {{cflags_cc}} -c {{source}} -o {{output}}"
  ...
}

Step 3: Create BUILD.gn

Finally, add a BUILD.gn to the source root with the following content to specify the hello executable should be compiled by main.cpp:

1
2
3
4
5
executable("hello") {
  sources = [
    "main.cpp",
  ]
}

Step 4: Build the project

Firstly, use gn gen command to specify to generate ninja in the out/ directory.

1
gn gen out

Then execute ninja to build the project

1
ninja - C out

If you modify the gn settings in the future, you don’t need to re-run gn, ninja will automatically update the settings. It makes the incremental build easier.

Details of Ninja

GN depends on Ninja. Ninja uses build.ninja files to define build rules. Unlike metaprogramming in Makefiles, build.ninja is almost completely static. Dynamic generation depends on other tools, such as GYP or CMake.

build.ninja is equivalent to ninja’s makefile. A simple build.ninja file is as follows, divided into two parts: rule and dependency.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# part rull
cc = gcc
cflags = -g - c
rule cc
command = $cc $cflags $in - o $out
rule link
command = $cc $in - o $out
rule cleanup
command = rm - rf * .exe * .o
#part dependency
build func.o: cc func.c
build main.o: cc main.c
build app.exe: link main.o func.o
build all: phony || app.exe
build clean: cleanup

Like Makefile, In build.ninja you can add more rules like

• phony: You can create aliases for other targets. Such as build all: phony || app.exe.

• default: If you do not specify a target on the command line, you can use default to specify the default target.

• pools: In order to support concurrent jobs, Ninja also supports the pool mechanism, the same as using the -j parallel mode.

The Ninja build log is saved in the root directory of the build process or in the .ninja_log file in the directory corresponding to the builddir variable.

The ninja command is used as follows:

1
2
3
4
# compile
ninja
# help
ninja - h

Ninja vs. Make

GN and Ninja’s positioning is very clear, that is to achieve faster build speed.

The design of Ninja is to consider the defects of make. It is considered that make has the following points that cause the compilation speed to be too slow:

• Implicit rules, make contains many defaults

• Variable calculations, such as how compilation parameters should be calculated

• Dependent Object Computing

In Ninja, the description file should look like as follows:

• Dependencies must be explicitly stated, dependency description files can be generated for convenience

• Without any variable calculation

• No default rules, no default values

In view of this, you can basically think of Ninja as the most streamlined version of make.

Compared to make, Ninja adds the following features:

• If the build command changes, the build is re-executed.

• The dependent directories have been created before the build. If this is not the case, we need to generate the directories before executing the command.

• Each construction rule, in addition to executing the command, also allows a description, which is actually executed and printed instead of actually executing the command.

• The output of each rule is buffered, which means that the input content will not be mixed together in parallel compilation.

There are too many build tools. I personally think that make is mainly popular. It can perform various implicit derivations and is more flexible. Every command has output.

The main design purpose of Ninja is to significantly increase the compilation speed for large projects such as chromium. On the one hand, it removes various calculations and derivations, removes some time-consuming things that need to be calculated, leaving only simple and important parts. If you write the build.ninja file by yourself, it is more tedious, as it depends on other build tools to generate. On the other hand it only outputs a description each time, instead of the real command execution output, the real command execution and then running in the background, only warning and error messages will be displayed, which also increases its speed.

Here is an article that compares the performance between Ninja and make: http://hamelot.io/programming/make-vs-ninja-performance-comparison/

Conclusion

There are too many build tools. Sometimes it’s hard to choose which one is the best for our project. I personally think that make is verly popular. It can perform various implicit derivations and is more flexible. Every command has output.

But if you build a large project, GYP and GN should be your first choice. Especially GN, as it is much faster than GYP. Under GN, Ninja even provides a pool mechanism to support concurrent jobs, the same as using the -j parallel mode of make.

The latest Google Chromium project has been migrated to GN from GYP.